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ABSTRACT 

A brief overview is provided of the Schools and 
Staffing Survey (SASS) program of the National Center for Education 
Statistics, and areas of research or survey evaluation in which work 
is ongoing or planned are described. The SASS is an integrated system 
of surveys of public and private schools, school districts, school 
principals and administrators, and teachers. The program provides 
information on teacher supply and demand, the workforce, 
characteristics of teachers and other school personnel, and data on 
school conditions and programs. The following issues are discussed: 
(1) frame and coverage; (2) questionnaire design; (3) data 
collection; (4) estimation and non-response; (5) measurement; and (6) 
evaluation of survey data. Research commitments in these areas will 
help provide a deeper understanding of SASS data, (Contains 10 
references , ) (SLD) 
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Iptroduction 

Until relatively recently, data on the workplace 
and main workforce of our education system, 
schools, teachers, and administrators, have not 
been available on a regular basis. The Schools 
and Staffing Survey(SASS) was designed to meet 
the need for information on the chaiacteristics and 
experience of teachers and administrators, to 
describe the essential features of the school as a 
place to work and a place to learn, and to provide 
data on aspects of teacher supply and demand, and 
attrition. The SASS was first fielded in the 1987- 
88 school year, was repeated in the 1990*91 
school year, and is intended to be conducted every 
three years. 

The SASS is a complex undertaking, both in 
design and in implementation. Becauise of its 
complexity and the fact that it is a regular 
recurring program of the National Center for 
Education Statistics(NCES), the SASS staff realize 
that opportunities exist to learn from the successes 
and failures of each collection cycle. These 
lessons provide the opportunity to make 
improvements to the program as well as gather 
information on the quality of the survey's data 
products. 

This paper is intended to serve two purposes: 1) 
to provide a brief overview of the SASS program; 
and 2) to identify areas of research or survey 
evaluation in which work is ongoing or planned. 



Overview of the SASS 

The SASS is an integrated system of surveys of 
public and private schools, school districts,sdiool 
principals and administrators, and teachers. The 
data collection program consists of seven mail- 
out/mail-back surveys implemented during one 
school year, followed by a mail survey of a 
subsample of teachers one year later. These 
surveys include: 

1. a survey of public school districtsOocal 
education agencies); 

2. a survey cf schools, public and private; 

3. a survey of school administrators/principals in 
the public and private sectors; 

4. a survey of teachers in the public and private 
sectors. 

iu iiiu ochool year rollowinij the implcmeuULion of 
the SASS, a subsample of teachers in the SASS 



teacher survey are selected to be included in the 
SASS Teacher Followup Survey. This mail 
survey, a survey of public and private school 
teachers, was designed to provide mformation on 
teadier attrition and retention in public and private 
schools. 

The SASS program has also included a research 
sample as part of its ongoing operations, thus 
providing opportunities to study questionnaire 
design, content, operational and survey methods 
issues in &e context of a large scale operation. 

The SASS is a broad multi-dimensional program, 
guided by four principal objectives: 

1 . to provide data on the components of teacher 
su[^ly and demand, shortages and turnover, and 
the policies and practices influencing supply and 
demand. 

2. to provide data on the principal/administrator 
workforce, including demographics and economic 
characteristics as well as their ac&demic 
background, qualifications, and experience. 

3. to provide data on teachers, including 
demographic characteristics, academic background, 
qualifications and experience, teaching 
assignments, workloads, and compensation. 

4. to provide data on school conditions and 
programs, including enrollments, staffing, 
organization, teaching load, problems and locus of 
control. 

The SASS accomplishes these objectives through 
a design that allows the development of state and 
national estimates for public schools and affiliation 
and national estimates for private schools. 
Schools are the primary sampling unit, and a 
sample of teachers, on average between four and 
dght, is selected in each sample school selected. 
Public school districts are included in sample 
when one or more schools in the district are 
selected. The follow*: are sample sizes for the 
1990-91 SASS: 5400 iocaI education agencies; 
13,200 schools (9,900 publiJc and 3,300 private); 
13,200 school principals/administrators (9,900 
public and 3,300 private); and 65,200 teachers 
(56,000 public and 9,200 private). 

By tiie nature of its content and design, the SASS 
provides opportunities to address issues on 
education policy. The existing SASS data linkages 
among the levels of the education hierarchy - - 
teachers, principals, schools^and ?choo] districts - - 
and the potcnlial to add several more, namely 
students and parents, indicates the importance of 



the SASS in &e elementary tnd secondary 
education statistics program in the National Center 
for Education Statistics. With this in mind, this 
paper presents the current operational and research 
issues of the program. The topics covered are: 
l)frame and coverage issues; 2) questionnaire 
design; 3) data collection; 4) estimation and 
nonresponse; 5) measurraient issues; and 6) 
evaluation of f»urvey data. 

prame mn^ C iTyrange Issoes 

The sampling frames for the school component of 
the 1990-91 SASS are the "public school universe" 
of the NCES' Common Core of Datc(CCD) and 
the Private School Universe Survey(PSS). The 
CCD contains statistical information collected 
annually on all public elementary and secondary 
schools and school districts in the nation. State 
Education Agencies(SEA) compOe and transmit 
data they collect from schools and school districts 
into formats defined by NCES. Information they 
provide includes school or district name, address, 
school type, enrollment and student characteristics, 
and the number of classroom teachers. 

The Private School Universe Survey(PSS) is a data 
collection aimed at building an accurate and 
complete list of private schools in the U. S. The 
schools on the PSS come from a combination of 
private school lists and area frame searches. The 
PSS contains data on religious orientation, level 
and size of school, enrollment, number of 
graduates, and number of teachers employed. 

CCD - SASS Differences 
The 1990-91 SASS rqjresentcd one of the first 
uses of the CCD for sampling purposes. School 
data in SASS were collected using the definitions 
established in CCD. However, an understanding 
of the relationship of these definitions to 
principals' and administrators' understandings of 
our concepts w is limited. Furthermore, because of 
the time needed to edit several CCD variables and 
the time needed to draw and prepare the sample, 
the CCD was used to draw the 1990-91 

SASS sample. Consequently, a number of schools 
in the SASS sample reported teacher counts and 
student enrollment counts that differed from the 
CCD file. Tmiing, school reorgamzation,CCD 
misreports and definition misunderstandings may 
play a role. A project is underway to understand 
these differences by characterizing the schools 
with discrepant information. 

Evaluation of die Private School List Building 

The sampling frame for private schools is a 
combination of list and area frame samples, the 

latter being necessary to compensate for the 
known undciojverage ot Uie list frame. The 
private school list frame is composed of private 



schools contained on a commercially available list 
from Quality Education Data(Q£D), private school 
associations, state departments of education, and 
other sources listing private schools. Recognizing 
the large undercoverage of the list frame in the 
previous PSS(i^proximately 20%), the 1991-92 
PSS made a substantial effort to acquire additional 
private school lists fix>m the states. A project to 
evaluate this effort and the impact of these new 
sources is undnway. 

EvahiatioB of Ibe Teacher listing 
The sampling of tiie teacher component of the 
SASS requires schools in the SASS sample to 
provide a list of teachers in their school along 
^th the following information: >\iiether new or 
experienced; race/ethnicity; bilingual/English as a 
Second Language(ESL); field of teaching. The 
issue for the SASS program is whether the school 
is filling the teacher list properly. SASS data have 
shown an inconsistency in the number of teachers 
listed by the schools during the listing operation 
eariy in the school year versus the numbers 
provided later in the year on the school 
questionnaire. A study is being developed that we 
expect will improve our insight as to how teacher 
estimates can be improved. This study will 
in\*olve a reinterview of the person providing the 
teadier lists. Reconciliation of differences in the 
original and reinterview list will provide an 
approximate idea of the "true** number of teachers. 
Also, under consideration is a reinterview and 
reconciliation of responses from the teacher listing 
operation and the school questionnaire. 

SASS Student Sample 

In recent years, interest has grown in augmenting 
the SASS program with a sample of students. 
Because SASS is a national probability sample of 
schools, SASS is capable of providing a national 
' probability sample of students distributed across 
elementary and secondary grades. Other NCES 
school-based surveys do not yield such samples, 
because tiiey are oriented to one or two grade 
levds. This sample could lead to the study of 
equity issues: wfaidi students are taught by 
better/worse prepared teachers? Which students 
are participating in various programs? The 
statistical issues with the student sample are the 
development of procedures to draw a sample of 
students at the school and the ability to correctly 
calculate a probability of selection. 

During the 1990-91 SASS, a research panel was 
fielded to address the issue of how a student 
sample should drawn - whether by an employee 
of the school or by the Census Bureau. 
Frazier(1992) reported on the results of this test 
and found that it was difficult for an untrained 
school employee lo coiiLJCily draw the sauiple. 
This field test also showed that the questions 
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aimed at determining th? probability of selection 
did not work well. Thus, NCES and Census 
Bureau staff continue to woric on this problem 
with a view to iriiplementing another pretest in 
early 1993. 

Expand Survey Covenge 
The SASS target population is limited in two 
vrays. First, schools offmng onW kindergarten 
and prekindergarten classes are denned to be out- 
of-scope in the SASS. This is a serious limitation 
given the current strong interest in policy issues 
related to early childhood education . 
Furthermore, NCES has no sampling frames that 
adequately cover the prekindergarten and 
kindergarten programs in die public and private 
sectors. I>uring the next year, NCES plans to 
study alternative ^)proaches to improving the 
coverage of prekindergarten and kindergarten 
programs. The adequacy of different sources of 
information on these programs will be reviewed 
and assessed. An exploration of the possibility of 
using the area search and area sampling 
approaches used in the PSS will also be reviewed. 
Second, the American Trust Territorie8(American 
Samoa, Guam, Virgin Islands, Northern Marianas, 
and Puerto Rico) are included in the CCD 
universe, but not in the SASS. Staff will explore 
the feasibility and cost of expanding SASS 
coverage to include the territories. 

Questionnaire Desipn 

SASS is a syctem of mail surveys v^th telq)hone 
follow-up. The surveys require teachers, 
principals, personnel officers, and administrative 
assistants to be able to respond to questions about 
complex concepts without much help. The 
complexity of the current forms and conc^ts offer 
many opportunities for the respondent to make 
mistakes. Several projects are now underway to 
address the shortcomings of the design and format 
of the current questionnaires. 

Cognitive Researdi Plrogram 
Results from the reinterview program at the 
conclusion of the 1988 cycle of the SASS 
indicated a number of items required 
improvement(Bushery, Royce, and Kasprzyk, 
1992). The program^s response was two fold: 1) 
to commit a substantial amount of professional 
time to reviewing completed questionnaires) and 2) 
to conduct a cognitive research program in 
preparation for the 1990-91 SASS. Similariy, in 
preparation for the next cycle of SASS^ a 
research program of detailed, probing interviews 
using the public school questionnaire was initiated 
in the fall of 1991. Jenkins(1992) rqx>rts on 
result? of the latter shidy. This program of 
cognitive research will next focus ou two 
questionraires. First, the SASS student 



questionnaire will be reviewed and redesigned for 
testing m the spring of 1993. Second, the school 
questionnaire, already the focus of much review, 
will be redesigned, n&formatted, and tested. 

Compiiter Assistod Interviewing 
The increasing availsAility of personal computers 
in sdiools and school districts suggests a potential 
^plication of computer assisted interviewing in 
tile SASS. Since SASS is a mail survey, a 
suggestion has been made to consider sending 
diskette-based data collection instruments to 
schools and school districts as an alternative to the 
p^>er and pencil instruments. This data collection 
system^s potential for improving the data collected 
in SASS is significant. Automated range checks, 
edits, logical edits, and skip pattern checks 
provide opportunities to clarify r^rted data at 
thdr source — the respondent — immediately upon 
reporting. Plans are being developed to design, 
implement, and evaluate an automated data 
ooUection instrument. 

Teacher*! Sdf-Report of Academic Background 
The 1987-88 SASS provided evidence of teachers 
having difficulty reporting their academic 
backgrounds. In response, two views of collecting 
these data have been proposed - by asking for the 
number of credits earned in critical suoject areas 
or by asking for the number of courses taken in 
eadi critical subject area. During the 1990-91 
SASS, a small research panel, 200 schools, having 
a sample of 867 teachers was fielded to test these 
two i^proaches. As part of the test, we asked for 
a signed authorization from each teacher to allow 
the NCES to obtain transcripts from the schools 
he/she attended. Transcripts will be coded bv 
subject area to allow comparisons to the self* 
reported data. 

Data Collection 

Several issues have arisen concerning the 
operation of the SASS data collection system. 
These issues bear on the quality of the reported 
data and the improvement of die timing of the 
availability of the SASS sample. 

Dttta Collection Mode 

SASS was designed to be primarily a mail* 
Ottt/mail-back survey. Telephone follov^p was 
used for all sample units not returning the mail 
questionnaire. Because there is a substantial 
telephone folIowup( 33% for the public schools 
and 46% for the private schools), there is concern 
about possible response bias due to the mode. 
Parmer, Shen, and Tan(1992) address the issue of 
possible response bias by mode. 

Ixdproving the Avauiabtiity of the ^aSS Sample 
Teacher sampling for the SASS requires the 
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development of a teacher list for each sampled 
school, lliese lists of teachers are requested firom 
the schools in the SASS sample, checked in at the 
regional office for completeoess, mailed to 
Jeffersonville v^ere the total numb^ of teachers 
of each type are keyed and transmitted to 
Washington. Washington then sends sampling 
instructions back to Jeffersonville where the 
specific teachers are selected. The data for the 
selected teachers are keyed and transmitted back 
to Washington. Washington then matches the 
teacher information to the school information to 
create the sample file. The sample file is then 
used to mail questionnaires to the teachers. 

This cumbersome process has an obvious and 
direct bearing on the timeliness of &e availability 
of the teacher sample. A working group has been 
established to study the potential for improving the 
efficiency of this operation. 

Data Comparability ProJ^ 
Response burden is a concern for all federal data 
collections. The hierarchical design of the SASS 
and a number of individuU items, particularly as 
they relate to school district staffing, have proven 
burdensome to SASS respondents. In response to 
the reactions of several school districts and in 
pursuit of better data collection methods, NCES 
developed a project to test whether state education 
agencies have the capacity to provide data from 
their automated record systems that would 
otherwise be collected in SASS from local 
education agencies. How comparable are the data 
available from the state education agencies to the 
data collected in the SASS from the individual 
local education agencies? Blank(I992) reports on 
the results of this project and its direction in the 
future. Successful collection of district level 
staffing items from a state automated record 
system would lead to major rethinking of SASS 
data collection methods for the teacher demand 
and shortage survey. 

Estimation and Nonresponse 

While unit response rates in the SASS are quite 
good, nonresponse remains a concern because of 
the hierarchical nature of the SASS design. 
Principals may often act as gate-keqxrs for the 
teachers in sample by not providing lists of the 
teachers in their schools; principals may complete 
the principal/administrator form but not the school 
form. Districts may also serve tt> gate-keq)ers for 
their schools. 

Nonresponse 

A project to investigate the characteristics of 
nonrespondents in the 1990-91 SASS is under 
Je-TcIupni-jnt. Cbdiacteristics ol SASS units - 
districts, schools, principals/administrators, and 



teachers - respondents and nonrespondents will be 
compared across many dimensions with a view to 
providing an understanding of nonresponse in the 
SASS. 

Work has ako begun on assessing the nonresponse 
adjustment cells chosen for the SASS school 
survey and the associated cell-collapsing strategy. 
These cells had been selected based on intuitive 
analytic judgement. This study, however, is 
/ intended to quantify these judgments and propose 
alternatives if necessary. Some preliminary results 
for the school component of the SASS are 
available inShen, Parmer,and T;m (1992). A 
replication of the study on other 'SASS analytic 
units - principals/administrators, ^hool districts, 
and teadiers • is also desirable. 

Plans are underway to increase the number of 
foUowups in the Teacher FoUowup Survey 
component of the SASS, thereby making this 
component a multiwave longitudinal study of 
teachers. Even though item nonresponse is 
relatively smaU in this survey, research on 
imputation methods that use previously collected 
data is desirable. 

Variance Estimation 

In SASS, the sampling unit is the school. School 
districts are brought into the sample because a 
school in the district has been selected in the 
SASS sample. Thus, the school district collection 
unit is an aggregation of schools(the sampling unit) 
belonging to the district. Kaufman(1992) 
addresses the issue of how well balanced half- 
sample r^lication methods estimate variances 
when the collection unit (school districts) is an 
aggregation of sampling units (schools). 

Generalized variance models provide data users an 
easy way of obtaining variance estimates for 
complex sample surveys. A project is under way 
to develop generalized variance models for each 
component of the SASS. 

Mcasnnsment Issocs 

Much attention has been given to resources and 
students as the principal measures of institutional 
improvement expenditures per pupil, for 
example. Statistics such as these, however, 
provide little information about school quality or 
tile quality of the educational experience in 
schools. To remedy this situation, a number of 
ideas, as discussed in the report of a Panel on 
Education Indicators (}99l), will be developed as 
potential measures of educational experience and 
mstitutional quality. A series of research projects, 
field studies, and feasibility studies will be 
developed during the next several yeaib. The 
research will be a combination of conceptual 
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research (appropriate measures), item and content 
research (the possible ways the measures can be 
implemented), and field and validation research 
(do these measures work in practice and do they 
work in large scale surveys). Originally 
conceived as a component of ibt SASS, the 
direction of the research may dtber lead to a new 
survey or significant modifications to an existing 
survey. 



Evaluation cf Survey Data 

Evaluation of the quality of survey data can take 
several approaches. Microdata studies vMch 
evaluate the quality of the individual response, 
field performance statistics, experiments to test 
competing methodologies, and macrocomparisons 
vnih other established data sources are all used to 
esUblish the validity of a daU set. SASS has 
several projects, ongoing and in the planning 
stages, which v/ill bring information to bear on the 
quality of the SASS data. 

SASS Reinterview Program 
SASS has fielded a reinterview program in each 
cycle. Each reinterview was aimed at measuring 
simple response variance, a measure of the 
inconsistency between responses over repeated 
applications of the question. Thus, the purpose of 
the reinterview was to identify questions needing 
improvement in the next cycle of SASS. Bushery, 
Royce, and Kasprzyk(1992) describe results from 
the reinterview programs and show how these 
programs can be a tool for identifying problem 
items in a questionnaire. While the Bushery et al 
paper shows results from a limited number of 
questions in both the 1987-88 and 1990-91 SASS, 
an analysis of the quality of substantially more 
items is available in an internal 
memorandum(Royce, 1992). 

The 1991-92 Teacher FoUovwip Survey(TFS) 
component of the SASS also conducted a 
reinterview to ascertain the quality of individual 
items. This reinterview program, however, 
featured the use of a probing, reconciled 
interviews to improve the reintervicw's diagnostic 
power. In this case, we expject to obtain 
mformation not only on questions tihat are 
unreliable, but also on the reasons for the 
inconsistency in responses. 

Comparing Estimates across Forms 

The SASS obtains the same or similar data across 
several survey forms. Thus, for example, it is 
possible to obtain rates of attrition and sepiuration 
not only in the SASS but also in the TPS. 
National counts of teachers are available from the 
teacher, bchool, and district forms. Estimates of 
the number of certified teachers are available from 



both the teacher and district questionnaires* The 
relationship between these seemingly equivalent 
estimates is not well understood. During the next 
year, a project will begin to identify all estimates 
of the same phenomena across the different 
components, to quantify the differences if they 
exist, and to try to understand the reasons for the 
differences. 

Evaluating Self-Reports of Uibanidty 
In both the 1987-88 and 1990-91 cycles of SASS, 
tibe question, "Which best describes the community 
in which this school is located?" was asked of the 
principal(for the administrator/principal survey) 
and the respondent to the school survey. The 
response categories were given as rural, small city 
or town, medium-sized city, suburb of a medium 
city. etc. These reports are highly subjective and 
have exhibited moderate response variance as 
determined through the reinterview 
program(Bushery et al, 1992). Recently 
Johnson(1989) developed a methodology for 
assigning "type of locale" codes based on the 
school mailing address being matched to Bureau of 
the Census ^ta files containing population and 
population density data, Standard Metropolitan 
Statistical Area(SMSA) codes, and a Census code 
defining urban and rural areas. A study is in 
progress to compare codes derived through the 
Johnson algorithm to the self-reported 
classifications found in the two cycles of SASS. 
These comparisons will give us a better 
understanding of this survey item. Since the self- 
report method is used on many NCES surveys, the 
results of this study have wider applicability than 
the SASS. 

SASS Quality Profile 

Work on developing a quality profile for the SASS 
has begun. The quality profile v^ll summarize 
methodological and evaluation research related to 
the SASS and will provide an overview of 
procedures for all phases of the survey - sample 
selection, datA collection, daU processing and 
estimation. It is intended to provide an overview 
of what is known about the sources and magnitude 
of errors in the SASS, and thus a sourcebook of 
information on the quality of the SASS data. 

SASS Uaer Survey 

In designing the SASS approximately six years 
ago, NCES anticipated a variety of users - 
education planners, policymakers, managers, 
government analysts, and academic researchers. 
By the end of 1992, SASS will have released 
several major reports, a number of E.D. Tabs, 
public use microdata tapes and CD ROMs, and 
restricted access data tapes. In February 1992, 
4e SASS Review Board, a working group of 
researchers intcicstcd in the use and evaluation of 
the SASS, suggested the need for a SASS user 
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survey. The puqx>8e of the survey would be to 
identify uses/users of the SASS and to assess 
wh^er they are consistent with the uses and users 
as identified in tiie design j^ase* The survey 
would also attempt to assess vviiether the available 
survey products meet user needs and how 
dissemination efforts could be improved. In the 
near future after the goals of survey are more 
clearly defined, we anticipate developing such a 
survey. 

Data Analysis: Quantity and Quality of Teacher 
Labor Supply 

This year an analytic project that focusses on 
estimating the effects of compensation and other 
policy variables on the quantity and quality of the 
teacher labor supply will begin. This project will 
address: 1) the estimation of tbt external labor 
supply facing schools and the effect of 
compensation and other school*level variables on 
this supply; 2) the estimation of internal labor 
supply, i.e. the retention of the teaching woricforce 
and the effect of compensation and other policy 
variables on temporary and permanent flows in 
and out of the teachmg profession* While a 
decidedly substantive analytic project, this project 
is intended to also re-evaluate the vacancy matrix 
data on die 1987-88 Teacher Supply and Demand 
Questionnaire, data that were not released on the 
public use tapes due to response inconsistencies. 



Any large complex data collection raises numerous 
questions about methods and data quality. SASS 
is still in its infancy in terms of understanding and 
use. The 1990*91 SASS included a methods 
research panel to help assist in answering 
unresolved methods issues. The 1993-94 SASS 
will also include such a panel. This conunitment 
as well as the research commitment described in 
the projects above will provide a much deeper 
understanding of the SASS data as well as 
improve the quality of the survey operations* 
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